Detecting and Grounding Important Characters in Visual Stories

Characters are essential to the plot of any story. Establishing characters before writing a story can improve clarity and overall flow narrative. However, previous work on visual storytelling tends focus detecting objects in images discovering relationships between them. In this approach, not distinguished from other when they fed into generation pipeline. The result is coherent sequence events rather than character-centric order address limitation, we introduce VIST-Character dataset, which provides rich annotations, including textual co-reference chains importance ratings for characters. Based propose two new tasks: important character detection grounding stories. For both tasks, develop simple, unsupervised models based distributional similarity pre-trained vision-and-language models. Our together with these models, serve as foundation subsequent analysing generating stories perspective.